Information collection system and information center

ABSTRACT

Each of vehicles acquires its own position, and transmits an image of a nearby outside taken by an own camera and a piece of information of the own position. An information center communicatable wirelessly with the vehicles receives the images and the pieces of information of the vehicles&#39; positions transmitted by the vehicles, and recognizes pieces of first character information appearing in the received images, respectively. Based on the received pieces of information of the vehicles&#39; positions as well as at least either points of interest or areas stored as collection target information, the information center determines at least either points of interest or areas for the recognized pieces of first character information, respectively, and stores association information where the pieces of first character information are associated with the determined at least either points of interest or areas.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of priority of JapanesePriority Application No. 2016-088307, filed on Apr. 26, 2016, the entirecontents of which are hereby incorporated herein by reference.

BACKGROUND

1. Technical Field

The present disclosure relates to an information collection system andan information center.

2. Description of the Related Art

An information collection system for storing images taken by cameras ofvehicles has been known (for example, see Japanese Laid-Open PatentApplication No. 2013-239087).

According to Japanese Laid-Open Patent Application No. 2013-239087,images taken by cameras of vehicles are stored, and useful imagesselected from the stored images are provided to vehicles.

SUMMARY

According to one aspect of the present disclosure, an informationcollection system includes a plurality of vehicles. Each of the vehiclesincludes one or more processors configured to acquire a position of thevehicle, and transmit an image of a nearby outside taken by a cameraincluded in the vehicle and a piece of information of the position ofthe vehicle acquired when the image is taken. The information collectionsystem further includes an information center wirelessly communicatablewith the vehicles, respectively. The information center includes one ormore processors configured to receive the images and the pieces ofinformation of the positions of the vehicles from the vehicles,respectively, and recognize pieces of first character informationappearing in the received images, respectively. The one or moreprocessors of the information center are further configured to, based onthe received pieces of information of the positions of the vehicles aswell as at least either points of interest or areas stored as collectiontarget information, determine at least either points of interest orareas for the recognized pieces of first character information,respectively, and store pieces of association information where thepieces of first character information are associated with the determinedat least either points of interest or areas, respectively.

Other objects, features and advantages will become more apparent fromthe following detailed description when read in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 generally illustrates one example of a configuration of aninformation collection system;

FIG. 2 is a flowchart generally illustrating one example of aphotographing control process carried out by a vehicle (i.e., aphotographing control unit);

FIGS. 3A and 3B illustrate one example of a QR code appearing in aphotographed image, and one example of QR code read data (secondcharacter information);

FIG. 4 is a flowchart generally illustrating one example of a process (aQR code reading process) to acquire QR code read data (second characterinformation) from a QR code appearing in a photographed image;

FIGS. 5A and 5B illustrate an actual example of a way to determine a POIor an area for character information carried out by an informationcenter (a POI-and-area determination unit);

FIG. 6 is a flowchart generally illustrating one example of a process,from recognition of character information through generation ofassociation information carried out by the information center (acharacter information recognition unit, the POI-and-area determinationunit, and an association information generation unit);

FIG. 7 illustrates an actual example of an error correction processcarried out by the information center (a character informationcorrection unit);

FIG. 8 is a flowchart generally illustrating one example of a process,from error correction on character information through storingassociation information in a delivery information DB 339 carried out bythe information center (the character information correction unit and astorage process unit); and

FIG. 9 illustrates one example of a method to send delivery informationto a vehicle from the information center (an information delivery unit).

DETAILED DESCRIPTION OF EMBODIMENTS

According to Japanese Laid-Open Patent Application No. 2013-239087mentioned above, it is possible to prevent useless images from beingprovided to users. However, stored images are provided to users as theyare, and therefore, the users may be not able to use information shownin the images effectively. For example, an image provided to a user mayshow a POI (Point Of Interest) of a shop that a user is interested in, asignboard or a banner concerning an event for a certain area, and soforth. However, it may be difficult for the user to immediatelyunderstand in which type the shop is, in which area the event will beheld, and so forth. Especially, when an image is provided to a vehicle'sdriver or an occupant during traveling, it may be desired that theinformation indicated by a signboard, banner, or the like, shown in theimage, can be immediately understood by the driver or the occupant.Otherwise, the vehicle may actually be far away from the correspondingshop, area, or the like, when the driver or the occupants haveunderstood the information indicated by the signboard, banner, or thelike. Thus, according to the technology disclosed by Japanese Laid-OpenPatent Application No. 2013-239087, it may be actually difficult forusers to effectively use information appearing in provided images.

In consideration of the point, an object of the present disclosure is toprovide an information collection system with which it is possible tostore pieces of information appearing in photographed images taken bycameras of vehicles in such a manner that users can use the informationmore effectively.

Below, embodiments will be described using the accompanying drawings.

FIG. 1 generally illustrates one example of a configuration of aninformation collection system 1 according to an embodiment. Theinformation collection system 1 includes a plurality of vehicles 2 andan information center 3. The information center 3 and each of thevehicles 2 are bidirectionally connected via, for example, acommunications network 4 such as a cellular phone network. Below,description will be made assuming that the vehicles 2 have the same orsimilar configurations.

Each vehicle 2 includes a camera 21, a sensor 22, a microphone 23, anECU (Electronic Control Unit) 24, and a wireless communications device25.

The camera 21 photographs an outside near the vehicle 2 in a certainrange in response to an operation instruction from the ECU 24 (actually,a photographing control unit 242, described later). An image(photographed image data) taken by the camera 21 is transmitted to theECU 24 via an on-vehicle network such as a 1-to-1 communications line ora CAN (Controller Area Network), and is temporarily stored in a storageunit 245 described later.

The sensor 22 detects various states of the vehicle 2 (vehicle states).The sensor 22 includes a GPS sensor that detects the position of thevehicle 2, an angular acceleration sensor (i.e., a gyro sensor) thatdetects angular accelerations along the three axes of the vehicle 2(i.e., the forward and backward axis, the left and right axis, and thevertical axis), and so forth. Data (vehicle state data) detected by thesensor 22 is transmitted to the ECU 24 via an on-vehicle network such asa 1-to-1 communications line or a CAN (Controller Area Network).

The microphone 23 is installed in the vehicle cabin, and acquires soundsof occupants (including the driver) of the vehicle 2 in response to anoperation instruction from the ECU 24 (actually, a sound recognitionunit 244 described later). The sounds acquired by the microphone 23 aretransmitted to the ECU 24 via an on-vehicle network such as a 1-to-1communications line or a CAN (Controller Area Network).

The ECU 24 carries out control operations concerning acquisition ofvarious sorts of data to be transmitted to the information center 3, andtransmission of the acquired information to the information center 3.The ECU 24, for example, includes a microcomputer, and implementsvarious control processes by executing various programs stored in a ROM(Read-Only Memory). The ECU 24 includes, as functional units implementedas a result of a CPU (Central Processing Unit) executing the variousprograms stored in the ROM, a transmission and reception process unit241, a photographing control unit 242, a QR code reading unit 243, and asound recognition unit 244. Also, the ECU 24 includes, as a storage areaprepared in an internal nonvolatile memory, a storage unit 245.

The transmission and reception process unit 241 transmits signals to theinformation center 3 and receives signals transmitted from theinformation center 3, through the wireless communications device 25.

The transmission and reception process unit 241 transmits the positionof the vehicle 2 (for example, information such as latitude andlongitude information) to the information center 3 successively based ondata that is input from the GPS sensor included in the sensor 22.

Also, in response to receiving a signal to request to photograph anearby outside of the vehicle 2 using the camera 3 from the informationcenter 3, the transmission and reception process unit 241 sends therequest to the photographing control unit 242.

Also, in response to a transmission request from the photographingcontrol unit 242, the transmission and reception process unit 241transmits image data photographed by the camera 21 and correspondingadditional information (i.e., image additional information) to theinformation center 3. The image additional information includes the timewhen the photographed image is taken, the position of the vehicle 2, theangular acceleration information of the vehicle 2, and so forth,acquired when the photographed image is taken, camera information (i.e.,the position in the vehicle where the camera 21 is installed, theinstallation angle, the angle of field, and so forth, of the camera 21).

Also, in response to a transmission request from the QR code readingunit 243, the transmission and reception process unit 241 transmits QRcode read data and additional information corresponding to the QR coderead data (i.e., QR code additional information) to the informationcenter 3. The QR code additional information includes the time when thephotographed image corresponding to the QR code read data is taken, theposition of the vehicle 2, the angular acceleration information of thevehicle 2, and so forth, acquired when the photographed image is taken.

Also, in response to a transmission request from the sound recognitionunit 244, the transmission and reception process unit 241 transmitssound recognized data and additional information corresponding to thesound recognized data (i.e., sound additional information) to theinformation center 3. The sound additional information includes the timewhen the sound data corresponding to the sound recognized data isacquired, the position of the vehicle 2 acquired when the sound data isacquired, and so forth.

The photographing control unit 242 controls the camera 21. Actually,when the transmission and reception process unit 241 has received aphotographing request from the information center 3, the photographingcontrol unit 242 responds to the photographing request to transmit anoperation instruction to the camera 21. In more detail, thephotographing control unit 242 transmits an operation instructioncorresponding to the contents of the instruction included in thephotographing request (for example, timing to start and endphotographing, the position to photograph, and so forth). Below, withreference to FIG. 2, a process flow of the photographing control unit242 to control photographing will be described.

FIG. 2 is a flowchart generally illustrating one example of a controlprocess carried out by the photographing control unit 242. The processof the flowchart is repetitiously carried out every certain timeinterval, during, for example, a period of time starting from thecompletion of an initial process carried out after ignition ON (IG-ON)of the vehicle 2 and ending at ignition OFF (IG-OFF).

In step S102, the photographing control unit 242 determines whether thetransmission and reception process unit 241 has received a photographingrequest from the information center 3. When the transmission andreception process unit 241 has received a photographing request from theinformation center 3, the photographing control unit 242 proceeds to thestep S104. When the transmission and reception process unit 241 has notreceived a photographing request from the information center 3, thephotographing control unit 242 ends the current process.

In step S104, the photographing control unit 242 transmits an operationinstruction corresponding to the instruction contents of thephotographing request to the camera 21. Thereby, the camera 21photographs a situation near the vehicle 2 during the period of time forthe traveling zone according to the operation instruction.

In step S106, the photographing control unit 242 transmits atransmission request, to transmit the photographed image data of thecamera 21 to the information center 3, to the transmission and receptionprocess unit 241, and ends the current process. Thus, the transmissionand reception process unit 241 transmits the photographed image data andthe image additional information to the information center 3.

Returning to FIG. 1, the QR code reading unit 243 recognizes a QR codeappearing in the photographed image of the camera 21. Also, the QR codereading unit 243 analyzes the recognized QR code, to acquire QR coderead data (second character information). Below, with reference to FIGS.3A, 3B, and 4, an actual example of a QR code reading process of the QRcode reading unit 243, and a process flow in the QR code reading unit243 will be described.

First, FIG. 3A illustrates one example of a QR code appearing in aphotographed image, and FIG. 3B illustrates one example of correspondingQR code read data (second character information).

In the example of FIG. 3A, the front of a B department store 51, facingan intersection, appears in a photographed image of the camera 21. Then,approximately at the center of the photographed image, an advertisementthat includes a relatively large QR code 52 of the B department store 51appears. The QR code reading unit 243 recognizes the QR code 52appearing in the photographed image illustrated in FIG. 3A, analyzes theQR code 52, and acquires the QR code read data illustrated in FIG. 3B.

In the example of FIG. 3B, the QR code read data is text data concerningan event of the B department store 51. Actually, the QR code read datahas the contents “<B DEPARTMENT STORE EVENT> YOU CAN FIND DELICIOUSFOODS AND GOOD THINGS OF KYUSHU. SUMMER LARGE EXHIBITION OF KYUSHU FROMAUGUST XX (WED) THRU YY (MON)”.

Next, FIG. 4 is a flowchart generally illustrating one example of aprocess to acquire QR code read data (second character information) froma QR code appearing in a photographed image (a QR code reading process).The process of the flowchart is carried out, for a photographed image,for example, each time immediately after the photographed image of thecamera 21 is stored in the storage unit 245.

In step S202, the QR code reading unit 243 tries to recognize a QR codeof a photographed image acquired from the storage unit 245.

In step S204, the QR code reading unit 243 determines whether any QRcode appears in the photographed image. If a QR code appears in thephotographed image, the QR code reading unit 243 proceeds to step S206.If no QR code appears in the photographed image, the QR code readingunit 243 ends the current process.

In step S206, the QR code reading unit 243 analyzes the recognized QRcode to acquire QR code read data.

In step S208, the QR code reading unit 243 transmits a transmissionrequest, to transmit the QR code read data to the information center 3,to the transmission and reception process unit 241, and ends the currentprocess. Thus, the transmission and reception process unit 241 transmitsthe QR code read data and the corresponding QR code additionalinformation to the information center 3.

Returning to FIG. 1, the sound recognition unit 244 responds to apredetermined operation of a user to activate the microphone 23, carryout a sound recognition process on sound data acquired by the microphone23, and generate sound recognized data. Then, after acquiring the soundrecognized data, the sound recognition unit 244 transmits a transmissionrequest, to transmit the sound recognized data to the information center3, to the transmission and reception process unit 241. For example, whenthe user of the vehicle 2 has found useful information near the vehicle2, the user of the vehicle 2 performs the predetermined operation andmurmurs the contents of the useful information. As a result, the soundrecognition unit 244 uploads the sound recognized data (text data)corresponding to the murmur to the information center 3, and theinformation can be thus shared with other users.

The storage unit 245 temporarily stores a photographed image of thecamera 21 in response to an instruction from the photographing controlunit 242.

The wireless communications device 25 carries out wirelesscommunications with the information center 3 via the communicationsnetwork 4, and includes, for example, a DCM (Data Communication Module).

The information center 3 is installed remotely from the vehicles 2, andis configured to be able to bidirectionally carry out wirelesscommunications with the vehicles 2, respectively. The information center3 includes a wireless communications device 31 and an informationcollection apparatus 32.

The wireless communications device 31 can be any device for carrying outwireless communications with each of the vehicles 2 via thecommunications network 4.

The information collection apparatus 32 collects information concerningpredetermined POIs (hereinafter, referred to as “target POIs”) andpredetermined areas (hereinafter, referred to as “target areas”).Generally speaking, the information collection apparatus 32 generatesinformation (i.e., “delivery information”, or “association information”described later) concerning the target POIs and the target areas fromphotographed image data, QR code read data, and so forth, transmittedfrom vehicles 2 (actually, the corresponding transmission and receptionprocess units 241), and stores the generated delivery information.

Also, the information collection apparatus 32 acquires photographedimage data, QR code data, and so forth, for monitoring, from therelatively small number of vehicles 2 from among vehicles 2 travelingnear each target POI and each target area, for example, generates thedelivery information for monitoring, and stores the delivery informationfor monitoring for each target POI and each target area.

Then, if it is determined that the stored delivery information formonitoring for a target POI or a target area has a change more than apredetermined amount, an updating determination unit 323 (describedlater) determines to update the delivery information for the target POIor the target area. In response, the information collection apparatus 32acquires photographed image data, QR code data, and so forth, from therelatively large number of vehicles 2 traveling near the target POI orthe target area, for example, generates the latest delivery informationfor the target POI or the target area, and stores the generated latestdelivery information in a manner of adding the generated information orupdating the exiting information with the generated information. Thus,the delivery information for being actually delivered by an informationdelivery unit 331 (described later) for the target POI or the targetarea is updated.

Thus, the information collection apparatus 32 generates and stores thedelivery information for monitoring, and also, generates and stores thedelivery information for being actually delivered to vehicles 2 or thelike by the information delivery unit 331 through a transmission andreception process unit 321 (described later).

The information collection apparatus 32 includes, for example, one ormore computers. The information collection apparatus 32 includes thetransmission and reception process unit 321, a photographing instructionunit 322, the updating determination unit 323, an image acquisition unit324, an additional information acquisition unit 325, a characterinformation recognition unit 326, a POI-and-area determination unit 327,an association information generation unit 328, a character informationcorrection unit 329, and a storage process unit 330.

Also, the information collection apparatus 32 includes the informationdelivery unit 331, an image storage unit 332, an additional informationstorage unit 333, a POI-and-area information DB 334, a map informationDB 335, a web crawling DB 336, a sound recognition DB 337, an unfixedinformation DB 338, and a delivery information DB 339.

Note that, the information concerning the target POIs and the targetareas is previously stored in the POI-and-area information DB 334. Whenthe target POIs and the target areas have been updated (some have beenadded, some have been deleted, and so forth), the information concerningthe target POIs and the target areas is updated.

The transmission and reception process unit 321 transmits signals to thevehicles 2 and receive signals from the vehicles through the wirelesscommunications device 31.

When the transmission and reception process unit 321 has receivedphotographed image data and image additional information, thetransmission and reception process unit 321 sends information indicatingthis fact to the image acquisition unit 324, and stores the receivedphotographed image data and image additional information in the imagestorage unit 332 and the additional information storage unit 333,respectively.

Also, when the transmission and reception process unit 321 has receivedQR code read data and QR code additional information from a vehicle 2,the transmission and reception process unit 321 sends informationindicating this fact to the storage process unit 330.

Also, when the transmission and reception process unit 321 has receivedsound recognized data and sound additional information from a vehicle 2,the transmission and reception process unit 321 stores the soundrecognized data in the sound recognition DB 337 in a predeterminedformat, and stores the sound additional information in the additionalinformation storage unit 333.

Also, when the transmission and reception process unit 321 has receiveda transmission request from the photographing instruction unit 322, thetransmission and reception process unit 321 transmits the photographingrequests to vehicles 2 designated by the transmission request.

When the transmission and reception process unit 321 has received adelivery request from the information delivery unit 331, thetransmission and reception process unit 321 transmits the deliveryinformation to vehicles 2 designated by the delivery request, registeredsmartphones of the users of the designated vehicles, or the like.

The photographing instruction unit 322 transmits a transmission requestto the transmission and reception process unit 321 according topredetermined conditions, to transmit photographing requests to vehicles2 designated by the transmission request through the transmission andreception process unit 321.

For example, for each target area and each target POI, if the updatingdetermination unit 323 does not determine to update the deliveryinformation, the photographing instruction unit 322 transmits thephotographing requests to the relatively small number of vehicles 2selected according to a predetermined rule from among vehicles 2traveling near the target area or the target POI, periodically, forexample. As a result, the photographing instruction unit 322 finallyacquires the corresponding photographed image data, QR code read data,and so forth for monitoring, with which the above-mentioned deliveryinformation for monitoring is generated and stored.

On the other hand, if the updating determination unit 323 determines,based on, for example, the corresponding delivery information formonitoring stored, to update the delivery information for a target POIor a target area, the photographing instruction unit 322 transmits thephotographing requests to the relatively large number of vehicles 2selected according to a predetermined rule from among vehicles 2traveling near the target area or the target POI. As a result, thephotographing instruction unit 322 finally acquires the correspondingphotographed image data, QR code read data, and so forth for keeping thedelivery information in the latest state.

That is, the latest delivery information is generated using the thusacquired photographed image data, QR code read data, and so forth forkeeping the delivery information in the latest state, and is stored insuch a manner that the generated information is added or the exitinginformation is updated with the generated information.

Thus, the delivery information for being actually delivered by theinformation delivery unit 331 is updated for each target POI and eachtarget area.

The photographing requests include instruction contents such as timingsto start and end photographing, the places to photograph, and so forth.

The updating determination unit 323 determines whether to update thedelivery information stored in the delivery information DB 339. Forexample, the updating determination unit 323 periodically compares, in atime-series manner, the above-mentioned delivery information formonitoring stored in the delivery information DB 339, to acquire thedifference in the delivery information for monitoring between differenttime points, for each target POI and each target area. Then, theupdating determination unit 323 determines, based on the acquireddifference, whether to update the delivery information for the targetPOI or the target area.

The image acquisition unit 324 acquires the photographed image data fromthe image storage unit 332, and sends the photographed image data to thecharacter information recognition unit 326.

The additional information acquisition unit 325 acquires the imageadditional information corresponding to the photographed image dataacquired by the image acquisition unit 324 from the additionalinformation storage unit 333, and sends the acquired image additionalinformation to the character information recognition unit 326.

The character information recognition unit 326 recognizes characterinformation (first character information) appearing in the photographedimage acquired by the image acquisition unit 324 through a characterrecognition process.

The POI-and-area determination unit 327 determines a target POI or atarget area concerning the character information recognized by therecognition unit 326, based on the position of the vehicle 2 acquiredwhen the photographed image is taken included in the image additionalinformation, and so forth.

The association information generation unit 328 generates associationinformation where the character information (first characterinformation) recognized by the character information recognition unit326 is associated with the target POI and the target area determined bythe POI-and-area determination unit 327.

Below, with reference to FIGS. 5A, 5B, and 6, an actual example of aPOI-and-area determination process of the POI-and-area determinationunit 327, and a flow of a process starting from recognition of characterinformation and ending at generation of association information, carriedout by the character information recognition unit 326, the POI-and-areadetermination unit 327, and the association information generation unit328, will be described.

FIGS. 5A and 5B illustrate an actual example of a process of thePOI-and-area determination unit 327 to determine a target POI or atarget area concerning character information (first characterinformation). FIG. 5A illustrates an actual example of a photographedimage acquired by the image acquisition unit 324. FIG. 5B is a plan viewillustrating a situation where the photographed image of FIG. 5A isphotographed. In more detail, FIG. 5B illustrates, in a superposingmanner, lines and polygons corresponding to the respective planimetricfeatures such as the buildings and the road that are GIS (GeographicInformation System) data stored in the map information DB 335, as wellas the vehicle 2 and the angle-of-field information (the angle range ofphotographing in the left and right directions).

As illustrated in FIG. 5A, in the photographed image, a nearby situationcentering on the forward direction of the vehicle 2 appears includingthe road 53 on which the vehicle 2 is traveling, the A restaurant 54present along the road 53, and so forth. The banner 55 installed by theA restaurant 54 appears at a right end part of the paragraphed image,and character information “STEAK FAIR IS NOW BEING HELD” is printed onthe banner 55. Therefore, the character information recognition unit 326recognizes the character information appearing in the photographedimage. Then, the POI-and-area determination unit 327 determines a targetPOI or a target area concerning the character information thusrecognized by the character information recognition unit 326.

The POI-and-area determination unit 327 can recognize the position ofthe vehicle 2 acquired when the photographed image is taken, and thecorresponding camera information (i.e., the angle of field of the camera21), from the image additional information acquired by the additionalinformation acquisition unit 325. Thus, as illustrated in FIG. 5B, thePOI-and-area determination unit 327 can recognize the nearby situationof the vehicle 2 occurring when the photographed image is photographed,and identifies the appearing planimetric features such as the buildings,from the GIS data of the map information DB 335, the image additionalinformation, and so forth.

Also, the POI-and-area determination unit 327 can determine whether theplanimetric feature such as the shop or the building appearing in thephotographed image corresponds to a target POI or is present in a targetarea, by comparing the name, or the like, of the shop or the buildingappearing in the photographed image, with the target POIs, the targetareas, and so forth, stored in the POI-and-area information DB 334.

Thus, in the example of FIGS. 5A and 5B, the POI-and-area determinationunit 327 can determine that the character information printed on thebanner 55 relates to the A restaurant 54 adjacent to the banner 55.

Also, in the example of FIGS. 5A and 5B, the corresponding target POIcan be determined to be a restaurant according to the contents of thecharacter information. Therefore, it is also possible to use thecontents of the character information to determine the correspondingtarget POI.

Next, FIG. 6 generally illustrates one example of a flow of a processstarting from recognition of character information and ending atgeneration of association information carried out by the characterinformation recognition unit 326, the POI-and-area determination unit327, and the association information generation unit 328. The process ofthe flowchart is carried out, for example, each time when thetransmission and reception process unit 241 receives photographed imagedata and image additional information data from a vehicle 2. Also, theprocess of the flowchart is carried out, for each photographed image.

In step S302, the character information recognition unit 326 recognizescharacter information (first character information) appearing in thephotographed image.

In step S304, the POI-and-area determination unit 327 tries to determinea target POI and a target area for the character information recognizedby the character information recognition unit 326 based on the positionof the vehicle 2 where the photographed image is taken. It is alsopossible that, as described above, the POI-and-area determination unit327 tries to determine a target POI and a target area for the characterinformation, based on the corresponding image additional information(the position of the vehicle 2, and the angle of field of the camera21), the information concerning the target POIs and the target areasstored in the POI-and-area information DB 334, and the map information(GIS data) stored in the map information DB 335.

In a simpler way, it is also possible that the POI-and-areadetermination unit 327 tries to determine a target POI and a target areafor the character information based on the position of the vehicle 2where the photographed image corresponding to the character informationis photographed, and the information concerning the target POIs and thetarget areas stored in the POI-and-area information DB 334.

It is also possible that, as described above, the POI-and-areadetermination unit 327 tries to determine a target POI and a targetarea, using the contents of the character information.

In step S306, the association information generation unit 328 determineswhether the POI-and-area information DB 334 includes a target POI thathas high relevancy with the character information recognized by thecharacter information recognition unit 326. If the POI-and-areainformation DB 334 includes a target POI that has high relevancy withthe character information recognized by the character informationrecognition unit 326, the association information generation unit 328proceeds to step S308. If the POI-and-area information DB 334 includesno target POI that has high relevancy with the character informationrecognized by the character information recognition unit 326, theassociation information generation unit 328 proceeds to step S314.

In step S308, the association information generation unit 328 determineswhether the POI-and-area information DB 334 includes a target areahaving high relevancy with the character information recognized by thecharacter information recognition unit 326. If the POI-and-areainformation DB 334 includes a target area having high relevancy with thecharacter information recognized by the character informationrecognition unit 326, the association information generation unit 328proceeds to step S310. If the POI-and-area information DB 334 includesno target area having high relevancy with the character informationrecognized by the character information recognition unit 326, theassociation information generation unit 328 proceeds to step S312.

In step S310, the association information generation unit 328 generatesinformation (i.e., association information) where the characterinformation recognized by the character information recognition unit 326is associated with the target POI and the target area determined by thePOI-and-area determination unit 327, stores the generated associationinformation in the unfixed information DB 338, and ends the currentprocess.

In step S312, the association information generation unit 328 generatesinformation (i.e., association information) where the characterinformation recognized by the character information recognition unit 326is associated with the target POI determined by the POI-and-areadetermination unit 327, stores the generated association information inthe unfixed information DB 338, and ends the current process.

If it is determined in step S306 that the POI-and-area information DB334 includes no target POI for the character information, theassociation information generation unit 328 determines whether thePOI-and-area information DB 334 includes a target area having highrelevancy with the character information recognized by the characterinformation recognition unit 326, in step S314.

If the POI-and-area information DB 334 includes a target area havinghigh relevancy with the character information recognized by thecharacter information recognition unit 326, the association informationgeneration unit 328 proceeds to step S316. If the POI-and-areainformation DB 334 includes no target area having high relevancy withthe character information recognized by the character informationrecognition unit 326, the association information generation unit 328proceeds to step S318.

In step S316, the association information generation unit 328 generatesinformation (i.e., association information) where the characterinformation recognized by the character information recognition unit 326is associated with the target area determined by the POI-and-areadetermination unit 327, stores the generated association information inthe unfixed information DB 338, and ends the current process.

In step S318, the association information generation unit 328 stores thecharacter information recognized by the character informationrecognition unit 326 in the unfixed information DB 338, and ends thecurrent process.

Returning to FIG. 1, the character information correction unit 329carries out error correction on the character information included inthe association information stored in the unfixed information DB 338.This is because the character information recognized based on thephotographed images through the character recognition process mayinclude errors.

The character information correction unit 329 carries out, for example,probabilistic error correction in a general-purpose method. For example,it is possible to detect and correct errors that may occur in thecharacter information included in the association information, using anN-Gram model, an editing distance (Levenshtein distance), or the like,which is known elemental technology of statistical natural languageprocessing. It is also possible, for example, to use a database (i.e.,an OCR (Optical Character Recognition) similar character DB) concerningsimilar characters for which errors may likely to occur in OCR.

Also, the character information correction unit 329 detects and correctserrors that may occur in the character information included in theassociation information, using web crawling acquired data concerning atleast either the target POIs or the target areas included in theassociation information.

Actually, the character information correction unit 329 acquires webcrawling acquired data concerning at least either the target POIs or thetarget areas included in the association information, from the webcrawling DB 336. Then, the character information correction unit 329 candetect and correct errors that may occur in the character informationincluded in the association information, by extracting texts similar tothe character information from the acquired web crawling acquired dataconcerning at least either the target POIs or the target areas includedin the association information, and carrying out comparison (matching)between the character information and the extracted similar texts.

Note that, the web crawling acquired data stored in the web crawling DB336 is web information acquired through web crawling operations on eachPOI and each area, and is updated periodically to correspond to thelatest web information.

Also, the character information correction unit 329 detects and correctserrors that may occur in the character information included in theassociation information, using sound recognized data concerning at leasteither the target POIs or the target areas included in the associationinformation.

Actually, the character information correction unit 329 searches thesound recognition DB 337 for the sound recognized data acquired at thesame time when the photographed images corresponding to the characterinformation included in the association information are taken near thepositions where the paragraphed image are taken. Then, the characterinformation correction unit 329 can detect and correct errors that mayoccur in the character information included in the associationinformation, by extracting texts similar to the character informationfrom the thus acquired sound recognized data, and carrying outcomparison (matching) between the character information and theextracted similar texts.

The storage process unit 330 stores the association information forwhich the character information correction unit 329 carried out errorcorrection on the character information, in the delivery information DB339. Also, after receiving the information indicating the fact that theQR code read data and the QR code additional information have beenreceived from the transmission and reception process unit 321, thestorage process unit 330 stores the QR code read data in the deliveryinformation DB 339 in a predetermined format, and stores the QR codeadditional information in the additional information storage unit 333.

Below, with reference to FIGS. 7 and 8, an actual example of thecharacter information correction process of the character informationcorrection unit 329, and a flow of a process carried out by the storageprocess unit 330 starting from the character information errorcorrection and ending at storing association information in the deliveryinformation DB 339 will be described.

FIG. 7 illustrates an actual example of the error correction process ofthe character information correction unit 329.

As illustrated in FIG. 7, in this example, character information “STEAKFAIR IS NOW BEING HELD” printed on the banner 55 installed by the Arestaurant 54 illustrated in FIG. 5A is extracted from photographedimages photographed by three different vehicles 2. The correspondingthree pieces of association information where the A restaurant 54 thatis a target POI is associated with the corresponding three pieces ofcharacter information, respectively, are stored in the unfixedinformation DB 338.

Actually, the respective pieces of character information in therespective pieces of association information are: “STEAT FAIR IS NOWBEING HELD”; “STEAK FAI IS NOW BEING HELD”; and “STEAK FAIR IS NOW BEINGHELD”. As can be seen, the first and second pieces of characterinformation include errors, respectively.

As described above, the character information correction unit 329 cancarry out: (1) probabilistic error correction according to ageneral-purpose method; (2) error correction using web informationacquired for the target POIs or the target areas; and (3) errorcorrection using sound recognized data.

In particular, for example, concerning the error correction (2), thecharacter information correction unit 329 can extract a text “steak fairis now being held” similar to the corresponding character information,based on web crawling acquired data concerning (i.e., web informationacquired for) the A restaurant 54 that is the corresponding target POI.

In another example, concerning the error correction (3), if soundrecognized data having the contents “A RESTAURANT, IT SAYS THAT A STEAKFAIR IS NOW BEING HELD! I WANT TO HAVE MEAT” is included in the soundrecognition DB 337, as illustrated in FIG. 7, the character informationcorrection unit 329 can extract the text “steak fair is now being held”from the sound recognized data.

Therefore, the character information correction unit 329 carries outerror correction to correct the first and second pieces of characterinformation included in the corresponding pieces of associationinformation stored in the unfixed information DB 338 into “STEAK FAIR ISNOW BEING HELD”. Then, the storage process unit 330 stores theassociation information for which the errors have been thus corrected inthe delivery information DB 339 as the corresponding deliveryinformation.

Next, FIG. 8 is a flowchart generally illustrating a flow of a processof the character information correction unit 329 and the storage processunit 330 starting from the error correction of character information andending at storing association information in the delivery information DB339. The process of the flowchart is carried out each time when, forexample, the association information generation unit 328 storesassociation information in the unfixed information DB 338.

In step S402, the character information correction unit 329 acquiresassociation information from the unfixed information DB 338.

In step S404, the character information correction unit 329 tries todetect an error from character information included in the acquiredassociation information, in a general-purpose method (probabilisticerror correction). If an error is detected, the character informationcorrection unit 329 corrects the error.

In step S406, the character information correction unit 329 acquires theweb crawling acquired data concerning the target POIs and the targetareas included in the association information from the web crawling DB336. Then, the character information correction unit 329 tries to detectan error from the character information included in the acquiredassociation information, based on the web crawling acquired data. If anerror is detected, the character information correction unit 329corrects the error.

In step S408, the character information correction unit 329 determineswhether sound recognized data acquired near the place where thephotographed image corresponding to the character information includedin the acquired association information is taken at the same time isincluded in the sound recognition DB 337. If the sound recognized datathat satisfies the conditions is included in the sound recognition DB337, the character information correction unit 329 proceeds to stepS410. If no sound recognized data that satisfies the conditions isincluded in the sound recognition DB 337, the character informationcorrection unit 329 proceeds to step S412.

In the step S410, the character information correction unit 329 tries todetect an error from the character information included in the acquiredassociation information based on the acquired sound recognized data. Ifan error is detected, the character information correction unit 329corrects the error.

In step S412, the character information correction unit 329 determineswhether the accuracy of the corrected character information included inthe acquired association information is high. A criterion to determinewhether the accuracy is high can be determined from various viewpoints.For example, it is possible to increase the possibility to determinethat the accuracy is high, if step S410 has been passed through.

If the accuracy of the corrected character information included in theacquired association information is high, the character informationcorrection unit 329 proceeds to step S414. If the accuracy of thecorrected character information included in the acquired associationinformation is not high, the character information correction unit 329ends the current process.

In step S414, the storage process unit 330 stores the acquiredassociation information on which the character information correctionunit 329 has carried out the error correction, in the deliveryinformation DB 339, and ends the current process.

Returning to FIG. 1, the information delivery unit 331 delivers, tovehicles 2 near a target POI or a target area or the smartphones of theregistered users of the vehicles 2, the delivery information concerningthe target POI or the target area stored in the delivery information DB339 through the transmission and reception process unit 321. Thus, theinformation delivery unit 331 delivers the association information wherethe character information is associated with the target POI or thetarget area, as well as the QR code read data concerning the target POIor the target area, to the corresponding vehicles 2 or the like. Below,with reference to FIG. 9, an actual example of giving the deliveryinformation to a user, which is delivered by the information deliveryunit 331, will be described.

FIG. 9 illustrates one example of giving the delivery information to auser, which is delivered by the information delivery unit 331. FIG. 9illustrates an image shown on the screen of a navigation apparatus of avehicle 2 when the QR code recognized data (second characterinformation) acquired based on the QR code recognized from thephotographed image illustrated in FIG. 3A and the associationinformation acquired based on the character information (first characterinformation) recognized from the photographed image illustrated in FIG.5A have been delivered to the vehicle 2.

As illustrated in FIG. 9, the message that “STEAK FAIR IS NOW BEINGHELD” is displayed for the A restaurant 54 that is the target POI, andthe fact that “SUMMER LARGE EXHIBITION OF KYUSHU” is held is indicatedfor the B department store 51. That is, instead of providing thecharacter information and the QR code appearing in the photographedimage as they are, the first character information and the secondcharacter information can be provided as the information associated withthe POIs or the areas. Thus, the user can immediately understand therelevancy of the provided information with the POIs or the areas, andthus, the user can use the information more effectively.

Note that, normally, QR code read data includes information concerningthe corresponding target POI or target area.

Thus, according to the embodiments, the transmission and receptionprocess unit 321 of the information center 3 receives, from a pluralityof vehicles 2, images photographed by their cameras 21 and imageadditional information such as the positions acquired when thephotographed images are taken, and so forth.

The character information recognition unit 326 of the information center3 recognizes the character information (the first character information)appearing in the photographed images. Also, the POI-and-areadetermination unit 327 determines target POIs or target areas for thecharacter information recognized by the character informationrecognition unit 326 based on the positions acquired when thephotographed images are taken, and the information concerning the targetPOIs and the target areas stored in the POI-and-area information DB 334.

The association information generation unit 328 of the informationcenter 3 generates the association information where the characterinformation recognized by the character information recognition unit 326is associated with at least either the target POIs or the target areasdetermined by the POI-and-area determination unit 327, and the storageprocess unit 330 stores the generated association information in thedelivery information DB 339.

Thus, it is possible to store the association information where thecharacter information appearing in the photographed images is associatedwith the target POIs or the target areas determined for the characterinformation, and it is possible to provide the association informationto users.

Therefore, the users can immediately understand that the characterinformation is associated with the particular POIs or areas because theprovided association information is displayed on the displays of thevehicles or the like. As a result, the users can immediately go towardthe POIs or the areas nearby. Thus, the users can effectively use theprovided information.

Also, according to the embodiments, the character information correctionunit 329 of the information center 3 carries out error correction on thecharacter information included in the association information.Therefore, even if errors occur in character recognition carried out bythe character information recognition unit 326, it is possible to storethe more accurate association information in the delivery information DB339 because error correction is thus carried out. That is, it ispossible to provide information that is more accurate to users.

Also, according to the embodiments, the character information correctionunit 329 of the information center 3 carries out error correction on thecharacter information included in the association information based onthe web crawling acquired data concerning the target POIs and the targetareas included in the association information. Also, the characterinformation correction unit 329 of the information center 3 carries outerror correction on the character information included in theassociation information based on the sound recognized data concerningthe target POIs and the target areas. Thus, it is possible to carry outerror correction on the character information included in theassociation information, using the web information acquired for thetarget POIs and the target areas included in the association informationor the sound recognized data (for example, text data corresponding tomurmurs for providing information of the users of vehicles 2).Therefore, it is possible to precisely correct errors that may occur incharacter recognition carried out by the character informationrecognition unit 326, which errors may be difficult to deal with throughgeneral-purpose error correction technology (for example, for a casewhere error correction is carried out on character information thatincludes an intentional typographical error, character information thatincludes a coined word, or the like).

Also, according to the embodiments, the QR code reading unit 243 of eachvehicle 2 recognizes a QR code appearing in a photographed image, andanalyzes the QR code to acquire QR code read data (second characterinformation). Then, the storage process unit 330 of the informationcenter 3 stores the QR code read data in the delivery information DB339. Thus, the QR code read data acquired through analysis of thecontents of the QR code appearing in the photographed image is stored inthe delivery information DB 339, and it is possible to provide the QRcode read data to users. Also, in many cases, QR code read dataincludes, as illustrated in FIG. 3B, information concerning a POI or anarea, and therefore, it is possible to provide the QR code read data asinformation concerning the POI or the area. Also, for example, if aphotographed image is provided as it is to a user, the user himself orherself reads a QR code appearing in the photographed image using anapplication of a smartphone. In contrast thereto, according to theembodiments, the user himself or herself need not read a QR code usingan application of a smartphone, for example, and therefore, it ispossible to improve the convenience of the user.

Thus, according to the embodiments, a position of each vehicle isacquired, and an image of a nearby outside taken by a camera of thevehicle and a piece of information of the position of the vehicleacquired when the image is taken are transmitted from the vehicle to aninformation center that is communicatable wirelessly with the vehicle.The information center receives the images and the pieces of informationthat indicate positions of the vehicles, respectively, and recognizespieces of first character information appearing in the photographedimages, respectively. Based on the received pieces of information of thepositions of the vehicles as well as at least either points of interestor areas stored as collection target information, the information centerdetermines at least either points of interest or areas for therecognized pieces of first character information, respectively, andstores, in a storage unit, pieces of association information where thepieces of first character information are associated with the determinedat least either points of interest or areas, respectively.

Thus, the pieces of association information where the pieces of firstcharacter information appearing in the photographed images areassociated with the points of interest or the areas determined for thepieces of first character information are stored, and are provided tousers. Therefore, when the provided pieces of association informationare displayed on displays of the respective vehicles, or the like, theusers can immediately understand that the pieces of first characterinformation relate to the particular points of interest or area.Therefore, the users can actually go to the points of interest or areasnearby immediately, and thus, it is possible to effectively use theprovided information. Thus, it is possible to store the informationappearing in the photographed images taken by the cameras of thevehicles in such a manner that it is possible to more effectively usethe information.

The information center corrects errors in the pieces of first characterinformation included in the pieces of association information.

Therefore, even if errors occur when the pieces of first characterinformation are recognized from the images, the errors are corrected,and thus, the more accurate pieces of association information can bestored. Thus, it is possible to provide the more accurate information tothe users.

When the information center corrects errors occurring when the pieces offirst character information are recognized, the web information acquiredfor the at least either points of interest or areas is used. Therefore,it is possible to correct errors in the pieces of associationinformation using the web information acquired for the points ofinterest or areas included in the pieces of association information.Therefore, even if errors occurring when the pieces of first characterinformation are recognized are those which are difficult to correctthrough general-purpose error correction technology, i.e., in caseswhere the character information includes intentional typographicalerrors, the character information includes coined words, or the like, itis possible to precisely correct the errors.

Each vehicle or the information center recognizes QR codes appearing inthe images, and analyzes the recognized QR codes to acquire pieces ofsecond character information, respectively. Then, the vehicle or theinformation center stores the pieces of second character information.

Thus, the pieces of second character information acquired from analyzingthe contents of the QR codes appearing in the images are stored andprovided to the users. In many cases, the pieces of second characterinformation include information concerning the corresponding points ofinterest or areas. Therefore, it is possible to provide the informationas information concerning the particular points of interest of areas tothe users. For example, if a photographed image is provided to a user asit is, the user himself or herself reads the QR code appearing in thephotographed image using a corresponding application, or the like. Incontrast thereto, according to the embodiments, the user is free fromsuch work of operating a corresponding application to read a QR code byhimself or herself. Thus, it is possible to improve the convenience ofthe users.

Thus, the information collection systems and the information centershave been described in the embodiments. However, the present disclosureis not limited to these embodiments. Various modifications andimprovements can be made within the scope of the present disclosure.

For example, in the above-described embodiments, the POI-and-areadetermination unit 327 tries to determine both target POIs and targetareas for the character information. However, it is also possible thatthe POI-and-area determination unit 327 tries to determine either targetPOIs or target areas for the character information. In this case, it issufficient that the POI-and-area information DB 334 stores informationconcerning either targets POI or target areas. Also, the associationinformation generation unit 328 associates the character informationrecognized by the character information recognition unit 326 with eitherthe target POIs or the target areas determined by the POI-and-areadetermination unit 327.

Also, in the above-described embodiments, the QR code reading unit 243is included in each of the vehicles 2. However, it is also possible thatthe QR code reading unit 243 is included in the information center 3, asshown in FIG. 1 indicated by a block 243 of a broken line.

Also, in the above-described embodiments, the character informationcorrection unit 329 is used. However, the character informationcorrection unit 329 can be omitted. In this case, the storage processunit 330 simply stores the association information generated by theassociation information generation unit 328 in the delivery informationDB 339 as delivery information as it is.

According to the embodiments, it is possible to provide informationcollection systems with which it is possible to store informationappearing in photographed images taken by cameras of vehicles in such amanner that users can use the information more effectively.

What is claimed is:
 1. An information collection system comprising: aplurality of vehicles, each vehicle of the vehicles including one ormore processors configured to acquire a position of the vehicle, andtransmit an image of a nearby outside taken by a camera included in thevehicle and a piece of information of the position of the vehicleacquired when the image is taken; and an information center wirelesslycommunicatable with the vehicles, respectively, the information centerincluding one or more processors configured to receive the images andthe pieces of information of the positions of the vehicles from thevehicles, respectively, recognize pieces of first character informationappearing in the received images, respectively, based on the receivedpieces of information of the positions of the vehicles as well as atleast either points of interest or areas stored as collection targetinformation, determine at least either points of interest or areas forthe recognized pieces of first character information, respectively,store, in a storage unit, pieces of association information where thepieces of first character information are associated with the determinedat least either points of interest or areas, respectively, and providethe association information to one or more of the vehicles, wherein theassociation information is different from the pieces of first characterinformation.
 2. The information collection system according to claim 1,wherein the one or more processors of the information center are furtherconfigured to correct errors in the pieces of first characterinformation included in the pieces of association information.
 3. Theinformation collection system according to claim 2, wherein the one ormore processors of the information center are further configured tocorrect errors in the pieces of first character information included inthe pieces of association information based on web information acquiredfor the at least either points of interest or areas included in thepieces of association information.
 4. The information collection systemaccording to claim 1, wherein either the one or more processors of eachof the vehicles or the one or more processors of the information centerare further configured to recognize QR codes appearing in the images,and analyze the recognized QR codes to acquire pieces of secondcharacter information, respectively; and the one or more processors ofthe information center are further configured to store the pieces ofsecond character information in the storage unit.
 5. The informationcollection system according to claim 2, wherein either the one or moreprocessors of each of the vehicles or the one or more processors of theinformation center are further configured to recognize QR codesappearing in the images, and analyze the recognized QR codes to acquirepieces of second character information, respectively; and the one ormore processors of the information center are further configured tostore the pieces of second character information in the storage unit. 6.The information collection system according to claim 3, wherein eitherthe one or more processors of each of the vehicles or the one or moreprocessors of the information center are further configured to recognizeQR codes appearing in the images, and analyze the recognized QR codes toacquire pieces of second character information, respectively; and theone or more processors of the information center are further configuredto store the pieces of second character information in the storage unit.7. An information center wirelessly communicatable with a plurality ofvehicles, respectively, the information center comprising one or moreprocessors configured to receive images of nearby outsides taken bycameras of the vehicles, respectively, and pieces of information ofrespective positions of the vehicles acquired when the images are taken,respectively, recognize pieces of first character information appearingin the received images, respectively, based on the received pieces ofinformation of the positions of the vehicles as well as at least eitherpoints of interest or areas stored as collection target information,determine at least either points of interest or areas for the recognizedpieces of first character information, respectively, store pieces ofassociation information where the pieces of first character informationare associated with the determined at least either points of interest orareas, respectively, and provide the association information to one ormore of the vehicles, wherein the association information is differentfrom the pieces of first character information.
 8. An informationcollection system comprising: a plurality of vehicles, each vehicle ofthe vehicles including one or more processors configured to acquire aposition of the vehicle, and transmit an image of a nearby outside takenby a camera included in the vehicle and a piece of information of theposition of the vehicle acquired when the image is taken; and aninformation center wirelessly communicatable with the vehicles,respectively, the information center including one or more processorsconfigured to receive the images and the pieces of information of thepositions of the vehicles from the vehicles, respectively, recognizepieces of first character information appearing in the received images,respectively, based on the received pieces of information of thepositions of the vehicles as well as at least either points of interestor areas stored as collection target information, determine at leasteither points of interest or areas for the recognized pieces of firstcharacter information, respectively, and store, in a storage unit,pieces of association information where the pieces of first characterinformation are associated with the determined at least either points ofinterest or areas, respectively, wherein either the one or moreprocessors of each of the vehicles or the one or more processors of theinformation center are further configured to recognize QR codesappearing in the images, and analyze the recognized QR codes to acquirepieces of second character information, respectively; and the one ormore processors of the information center are further configured tostore the pieces of second character information in the storage unit. 9.The information collection system according to claim 8, wherein the oneor more processors of the information center are further configured tocorrect errors in the pieces of first character information included inthe pieces of association information.
 10. The information collectionsystem according to claim 9, wherein the one or more processors of theinformation center are further configured to correct errors in thepieces of first character information included in the pieces ofassociation information based on web information acquired for the atleast either points of interest or areas included in the pieces ofassociation information.